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(54) Apparatus, method and system for selective image processing 



(57) An apparatus for selectively applying image 
processing to an image, comprising: 

means for receiving an input signal corresponding 
to the image from an image source in a primary 
color domain; 

means for receiving a command selecting a sample 
of the input signal; 

means for receiving a command identifying a color 
correction parameter in the primary color domain 
associated with the sample; 



means defining a sector in the hue-domain about 
the hue-domain parameter associated with the 
sample; 

means defining a qualification curve corresponding 
to the sector; and a mixer operative for mixing the 
input signal and the color correction parameter with - 
in the sector in accordance with the qualification 
curve. 



means for determining a hue-domain parameter as- 
sociated with the sample; 
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Description 
TECHNICAL FIELD 

[0001] The present invention relates generally to im- 
age processing, and more particularly relates to an ap- 
paratus, a system and methods that allow the user of 
an image processing system to implement scene-by- 
scene color manipulation in the primary color domain to 
color correction regions of a video image isolated in the 
hue domain using hue, saturation, and luminance qual- 
ification. 

BACKGROUND OF THE INVENTION 

[0002] In a video signal color correction system, var- 
ious types of image processing are often employed to 
create, enhance, compress, filter, or otherwise modify 
characteristics of the video image. In certain types of 
video image processing systems, especially post-pro- 
duction color correction systems for motion picture film 
and/or video tape, color corrections are typically made 
on a scene-by-scene basis, A "scene" is a sequential 
collection of image, often shot from the same camera, 
having the same viewpoint, composed in a certain way, 
etc. An operator using a typical post-production color 
correction system observes a target frame of the scene 
on a video monitor, adjusts the color and other param- 
eters of the frame until it is aesthetically satisfactory, and 
stores the color correction parameters in system mem- 
ory. The color correction system preferably automates 
the application the stored color correction parameters 
to the other frames of the scene. 
[0003] For example, the system operator or "colorist" 
typically selects a scene to manipulate, and then selects 
a particular frame from the scene for manual manipula- 
tion. The colorist views the target frame as a still image 
on the system monitor and applies color corrections via 
a control panel to adjust the color parameters for a par- 
ticular region of the target frame. The color correction 
system typically allows the colorist to isolate a particular 
region of the frame and to alter the intensity of the pri- 
mary color components, red, green and blue, which in 
various combinations produce all of the colors that the 
system can produce. The colorist may than apply color 
corrections to another region of the frame, and so forth, 
until ail of the desired regions have been color corrected. 
The correction settings are then stored in system mem- 
ory. 

[0004] After the colorist is satisfied with the adjust- 
ments made to the target frame, the color correction sys- 
tem, which is typically computer-controlled, automati- 
cally applies the stored color corrections to each frame 
in the scene on a frame-by-frame basis. The color cor- 
rected frames of the scene are then recorded on film or 
videotape. The steps are repeated for other scenes in 
the film or video tape, often with different correction set- 
tings stored for different scenes. This process may be 



repeated as needed to create a color corrected master 
film or video tape that reflects multiple color adjustments 
to multiple frames in multiple scenes. In the general 
case, multiple color adjustments may be applied to all 
5 of the frames of amotion picture or other video program. 
[0005] Devices are known in the art for isolating a re- 
gion of a still image for applying one set of color correc- 
tions, with other regions of the image receiving a differ- 
ent set of color corrections. These devices allow a color 
correction system to isolate a particular region of a 
frame to receive special image processing. For exam- 
ple, when color correcting a soft drink commercial it may 
be desirable to isolate the beverage container from the 
remainder of the image. The color of the beverage con- 
tainer may then be enhanced to make the can stand out 
from the rest of the image. But applying color correction 
to each frame of a film or video tape individually is ex- 
tremely tedious and time consuming. Automating the 
process of applying the color correction parameters de- 
fined for one frame of a scene to the other frames is, 
therefore, highly desirable. 

[0006] Automatically isolating a color correction re- 
gion presents a difficult technical problem because the 
region of interest may change in size, shape, location, 
and/or geometry over the frames in a scene. That is, an 
object in a video scene typically moves over the frames 
of the scene. For example, consider a scene in which a 
bottle is lifted, tilted, moved toward a glass, turned to- 
ward the viewer, and then further tilted to pour the con- 
tents of the bottle into a glass. In this scene, the geom- 
etry of the bottle changes from a side view of the bottle 
(an irregular shape) to a top view of the bottle (an es- 
sentially round shape). The bottle also changes in loca- 
tion and size over the several frames of the scene. 
[0007] The state of the art in automated color correc- 
tion technology is somewhat lacking for a number of rea- 
sons. Video images are typically defined on a pixel-by- 
pixel basis by intensity levels of the primary colors, red, 
green and blue (R,G,B). In a typical color correction sys- 
tem, the primary colors are mixed in varying intensities 
to produce all of the colors that the system can produce. 
Color data in the R,G,B format, often referred to as the 
"primary color domain," may be linearly transformed into 
secondary color components, hue, saturation and lumi- 
nance, which is sometimes referred to as the "hue do- 
main." Certain prior art color correction systems apply 
color corrections in the primary color domain, whereas 
others apply color corrections in hue domain. 
[0008] Operating in the primary color domain has cer- 
tain drawbacks associated with isolating color correc- 
tion regions. In a color video image, the intensity of the 
red, green, and blue components for a particular object 
depicted in a scene can vary from frame to frame in re- 
sponse to the brightness, or shadowing, of the item. In 
addition, many different colors may have the same in- 
tensity of one of the particular color components. For 
this reason, it is difficult to isolate a particular object in 
a scene by focusing on the intensity of the red, green, 
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and blue constituents of the pixels of the object. In other 
words, isolating a particular object, as that object moves 
from frame to frame within a scene., is difficult when an- 
alyzing the video data in the primary color domain. 
[0009] Color correction systems that operate in the 5 
hue domain have been developed to overcome the re- 
gion isolation drawbacks associated with color correc- 
tion system that operate in the primary color domain. 
For example, color correction systems have been de- 
veloped that allow an operator to select and manipulate 
a color correction region in the hue domain. These sys- 
tems typically include equipment for generating and po- 
sitioning a cursor on a video monitor to allow selection 
of a color correction region defined by the hue at the 
selected position. Circuitry responsive to the cursor lo- 
cation selects one of a plurality of color correction cir- 
cuits to become operative for directing secondary color 
correction parameters (i.e. : adjustments to the hue, sat- 
uration, and luminance) only to regions in the video im- 
age corresponding to the hue selected by the cursor. 
This type of system therefore allows application of sec- 
ondary color correction parameters to all regions of the 
image bearing the hue that was selected with the cursor. 
[0010] Color correction systems that operate in the 
hue domain, however, also exhibit certain drawbacks. 
First, they incur a relatively high processing overhead 
because the entire video image is typically transformed 
from the primary color domain to the hue domain. Color 
corrections are then applied in the hue domain, and the 
image is then transformed back from the hue domain to 
the primary color domain. Thus, the system performs 
two linear transformations on the entire video image 
even though no color corrections may be applied to 
large portions of the image. As a result, even those por- 
tions of the video image that are not color corrected must 
be transformed from the primary color domain to the hue 
domain, and then back to the primary color domain. 
[0011] Second, transforming color data from the pri- 
mary color domain to the hue domain, and then back to 
the primary color domain, can impart "color artifacts" in- 
to the video image. That is, applying a linear mathemat- 
ical operation to digital data, followed by the inverse 
mathematical operation, can alter the data for a small 
percentage of pixels. This phenomenon occurs due to 
the truncation of irrational number in floating-point arith- 
metic. Stated differently, due to the limited precision of 
linear mathematical operations applied to digital data, 
applying an arithmetic operation to digital data, followed 
by the inverse arithmetic operation, can alter the values 
for some of the data points. Although the vast majority 
of data points are not altered by the arithmetic opera- 
tions, a small percentage of data points are typically al- 
tered. These altered data points are known as "color ar- 
tifacts." 

[0012] To avoid the combined drawbacks associated 
with region isolation in the primary color domain and 
color artifacts created through operation in the hue do- 
main, color correction systems have developed that ap- 



ply color corrections in the primary color domain using 
geometric constraints to isolate color correction regions. 
For example, certain color correction systems include a 
video parameter control system operative for selecting 
a spatial region or window in a video image for color cor- 
rection. A track ball allows selection of a spatial region 
by dragging a cursor to draw a window around an object 
of interest in the picture. The operator then adjusts con- 
trols that affect only the selected window or region. A 
first set of correction signals may be stored for the se- 
lected region, and a separate second set of correction 
signals may be stored for areas of the picture outside 
the selected region, thereby allowing multiple sets of 
corrections for a given frame. Although these systems 
provide some control over the movement of the window 
over a plurality of frames in a scene, the region is es- 
sentially static and the geometry is invariably that of a 
box. 

[0013] Other prior art image processing computer 
software known as the "Simple Windows" and "Power 
Windows" features are extensions of the windows fea- 
tures of previous system. These features are provided 
in the RENAISSANCE 8:8:8™ digital color enhance- 
ment system, manufactured by the assignee of the 
present invention. In the Simple Windows feature, a win- 
dow is a predetermined regularly shaped area or region 
of the video image that can be varied in size. The colors 
within the window are independently adjustable from the 
colors of the rest of the image. Primary or secondary 
color enhancements can occur both inside and outside 
a window, and each adjustment is independent of the 
other. The Simple Windows feature entails use of a sim- 
ple geometric form — a square or rectangle — for a win- 
dow shape. A Simple window is always rectangular and 
is defined by four points. Lines defining the window are 
always straight vertical or horizontal. Furthermore, there 
are no soft edges, that is, there is a sharp delineation 
between the inside and outside of the window, which 
sometimes produces undesirable image effects at the 
boundaries. 

[0014] The more recent "Power Windows" feature 
provides more choices of the shape for the window, for 
example, circular, rectangular, half screen, split in the 
middle, etc. Using this system, the operator can select 
windows including multiple squares, multiple diamonds 
(essentially rotated squares), horizontal and vertical 
bars, circles, ellipses (a warped circle), and so forth. The 
sizes of these windows can be varied so long as the 
shape remains regular. Furthermore, a "soft edges" fea- 
ture is available to provide a gradual transition in color 
correction from the inside to the outside of a selected 
window. Power Windows, however, are still confined to 
regular shapes, with predefined geometry. 
[0015] The commonly owned patent application Seri- 
al No. 08/912,662, entitled "User Defined Windows For 
Selecting Image Processing Regions," filed August 18, 
1997 by Xueming Henry Gu, et al., describes further im- 
provements in the RENAISSANCE 8:8:8™ digital color 
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enhancement system. This system allows the user to 
select color correction regions corresponding to objects 
in a target frame using a geometric masking technique. 
For example, the user may mask the same object in the 
first and last frames of a scene. The system then auto- 
matically "morphs" or changes the shape of a selected 
object over the intervening frames of a scene. Although 
the system can automatically morph an object over a 
number of frames, the edges of the automatically- 
morphed object can diverge from the edges of the actual 
object in the scene. The system therefore includes a util- 
ity that allows the user to manually correct automatical- 
ly-morphed objects on a frame-by-frame basis. While 
this utility is quite effective for correcting unacceptable 
divergence, manually imparting the required adjust- 
ments on a frame-by-frame basis can be tedious and 
time consuming. 

[0016] In sum, there is a continuing need for a color 
correction system that allows the isolation of color cor- 
rection regions that move and change size and geome- 
try over several frames in a scene. There is a further 
need for a color correction system that isolates regions 
and implements color corrections without imparting 
color artifacts into the video image. And there is a need 
forfurtherfor automating the process of color correction 
region isolation to minimize or obviate the need for man- 
ual frame-by-frame adjustment of the processed video 
image. 

SUMMARY OF THE INVENTION 

[0017] The present invention overcomes the disad- 
vantages described above in a color correction system 
that implements primary color manipulation using sec- 
ondary color correction region isolation. As such, the 
system performs color correction in the primary color do- 
main, which means that color data that is not manipu- 
lated passes through the system unaltered. This aspect 
of the invention prevents color artifacts from appearing 
in the processed video image and lowers the data 
processing overhead because no operations, other than 
timed buffering to maintain synchronism, are performed 
for unprocessed or pass-through pixels. 
[0018] Although color correction is performed in the 
primary color domain, the system isolates color correc- 
tion regions through operations performed in the hue do- 
main. Specifically, hue qualification, saturation qualifi- 
cation, luminance qualification, and an optional alpha fil- 
ter are combined to define an alpha qualification func- 
tion that isolates a region or "hue sector" for color cor- 
rection. The alpha qualification function may have a 
shape in the hue domain that tapers or softens the ap- 
plied color correction towards the edges of the corrected 
hue sector. The system includes a user interface that 
allows the colorist to select the shape of the hue, satu- 
ration, and luminance alpha qualification functions, as 
well as the shape of the optional alpha filter, in the hue 
domain. This aspect of the invention gives the colorist 



fine control over the color isolation and correction proc- 
ess, including fine control over the sharpness of the 
color correction at the edges of color corrected hue sec- 
tors. 

5 [0019] Color correction region isolation in the hue do- 
main may be combined with geometric constrains. Ge- 
ometric constraints allow the colorist to limit the auto- 
matic application of color correction to a defined portion 
of the frame area, over the several frames of a scene. 
10 As a mechanism for selectively applying color correction 
to the frames of a scene, geometric constraints may be 
used to increases the system's ability to isolate discrete 
objects for color correction. For example, a colorist may 
use a geometric constraint to exclude from color correc- 
ts tion discrete objects that have a similar color to a target 
object. 

[0020] Generally described, the invention is an appa- 
ratus and a method for selectively applying image 
processing to an image. Color correction equipment re- 

20 ceives an input signal from an image source in a primary 
color domain. The color correction equipment also re- 
ceives a command selecting a sample of the input sig- 
nal, and a command identifying a color correction pa- 
rameter in the primary color domain associated with the 

25 sample. The color correction equipment determines a 
hue-domain parameter associated with the sample and 
a sector in the hue-domain about the hue-domain pa- 
rameter associated with the sample. The color correc- 
tion equipment then defines a qualification curve corre- 

30 sponding to the sector and mixes the input signal and 
the color correction parameter within the sector in ac- 
cordance with the qualification curve. The hue-domain 
parameter may be selected from the group including 
hue, saturation, and luminance. 

35 [0021] The color correction equipment may also de- 
termine a second hue-domain parameter associated 
with the sample and define a second qualification curve 
about the second hue-domain parameter and corre- 
sponding to the sector. The-f irst and second qualification 

40 curves are then combined to obtain a combined qualifi- 
cation curve. In this case, the color correction mixes the 
input signal and the color correction parameter within 
the sector in accordance with the combined qualification 
curve. The first and second hue-domain parameters 

45 may be selected from the group including hue, satura- 
tion, and luminance. 

[0022] In addition, the color correction equipmentmay 
determine a third hue-domain parameter associated 
with the sample and define a third qualification curve 

50 about the third hue-domain parameter and correspond- 
ing to the sector. The first, second, and third qualification 
curves are then combined to obtain a total qualification 
curve. In this case, the color correction equipment mixes 
the input signal and the color correction parameter with- 

55 in the sector in accordance with the total qualification 
curve. The first, second, and third hue-domain parame- 
ters may include the hue, saturation, and luminance. 
[0023] The color correction equipment may also apply 
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a filter to the total qualification curve to obtain an alpha 
qualification curve. If this case, the color correction 
equipment mixes the input signal and the color correc- 
tion parameter within the sector in accordance with the 
alpha qualification curve. Moreover, the color correction 
equipment may receive a command defining geometric 
constraint corresponding to the input signal, Inthiscase, 
the color correction equipment limitsthe mixing the input 
signal and the color correction parameter in accordance 
with the geometric constraint. 
[0024] The invention also provides a method 

for selectively applying image processing to a plu- 
rality of frames defining a scene. Color correction equip- 
ment receives an input signal defining the scene from 
the image source in a primary color domain. The color 
correction equipment also receives a command select- 
ing a target frame of the scene and another command 
selecting a sample of the target frame. The color cor- 
rection equipment also receives a command identifying 
a color correction parameter in the primary color domain 
associated with the sample. The color correction equip- 
ment then determines a hue-domain parameter associ- 
ated with the sample and defines a sector in the hue- 
domain about the hue-domain parameter associated 
with the sample. The color correction equipment also 
defines a qualification curve corresponding to the sec- 
tor. Then, for each frame of the scene, the color correc- 
tion equipment mixes the input signal and the color cor- 
rection parameter within the sector in accordance with 
the qualification curve. 

[0025] These and other features and advantages of 
the invention may be more clearly understood and ap- 
preciated from a review of the following detailed descrip- 
tion of the disclosed embodiment, and by reference to 
the appended drawings and claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0026] 

FIG. 1 is a schematic block diagram of a color cor- 
rection system constructed in accordance with an 
illustrative embodiment of the present invention. 
FIG. 2 is a schematic block diagram illustrating the 
use of an image processing system for implement- 
ing color correction Isolation using secondary color 
parameters. 

FIG. 3 (prior art) is a diagram illustrating the rela- 
tionship between the additive primary colors, a color 
wheel, and a color bar. 

FIG. 4 (prior art) is a diagram illustrating the rela- 
tionship between color parameters in the primary 
color domain and in the hue domain. 
FIG. 5 is a diagram illustrating a user interface for 
a color correction system. 

FIG. 6A is a diagram illustrating a portion of the user 

interface of FIG. 5 in greater detail. 

FIG. 6B is a diagram illustrating an alternative user 



interface for a color correction system. 
FIG. 7 is a logic flow diagram illustrating a method 
for operating a color correction system. 
FIG. 8 is a logic flow diagram illustrating a routine 
5 for automatically applying color corrections defined 
for a target frame of a scene to the other frames of 
the scene. 

FIG. 9 is a logic flow diagram illustrating a routine 
for computing a color correction qualifier for color 
10 enhancing a video scene. 

FIG. 10 is a schematic block diagram illustrating 
color correction equipment constructed in accord- 
ance with an illustrative embodiment of the present 
invention. 

FIG. 11 is a schematic block diagram illustrating 
color enhancement equipment that may form a 
component of the color correction equipment of in 
FIG 10. 

FIG. 12 is a schematic block diagram illustrating a 
color correction block that may form a component 
of the color enhancement equipment of in FIG 1 1 . 
FIG. 13A is a schematic block diagram illustrating 
a color mixer circuit that may form a component of 
the color correction block shown in FIG 12. 
FIG. 13B is a schematic block diagram illustrating 
a transformation matrix circuit that may form a com- 
ponent of the color correction block shown in FIG 
12. 

FIG. 14 is a schematic block diagram illustrating an 
index generator circuit that may form a component 
of the color correction block shown in FIG 12. 

DETAILED DESCRIPTION OF THE DISCLOSED 
EMBODIMENT 

[0027] The invention may be embodied a color cor- 
rection system that implements scene-by-scene color 
manipulation in the primary color domain to color cor- 
rection regions of a video image isolated in the hue do- 
main using hue, saturation, and luminance qualification. 
The invention may be implemented using programma- 
ble logic units (PLUs), field-programmable gate arrays 
(FPGAs), or other types of reprogrammable parallel- 
processing hardware. The system may be configured 
as a number of color correction blocks arranged in se- 
ries or parallel. Each correction block typically allows a 
coloristto define up to sixteen non-overlapping hue sec- 
tors with associated corrected color values and qualifi- 
cation curves. 

[0028] The color correction equipment processes a 
real-time video signal propagating at 74 mHz even 
though the color processing circuitry operates at 66 
mHz. This is possible because only a subset of the video 
signal needs to be passed through the color processing 
circuitry. In particular, each frame of video data includes 
a visible raster defining the image shown on a display 
device, and a non-visible portion including horizontal 
and vertical overscan areas, retrace intervals, and so 
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forth. Only the visible raster portion of the video signal 
needs to be passed through the color processing circuit- 
ry. The color correction equipment therefore strips the 
visible raster out of each frame and passes only the pix- 
els of the visible raster through the 66 mHz color 
processing circuitry. 

[0029] The non-visible portion of each frame bypass- 
es the color processing circuitry in an unprocessed input 
video data signal propagating at 74 mHz. Afterthe color 
corrected visible raster exits the color processing circuit- 
ry, the non-visible portion of each frame is recombined 
with the color corrected visible raster to construct a com- 
plete output frame of video data. Of course, the timing 
must be controlled so that the color-corrected visible 
raster reaches the combining circuitry at the same time 
as the unprocessed input video signal. Because the ra- 
tio of the data in the visible raster to the data in a com- 
plete frame is equal to or less than the ratio of the data 
propagalion rates (i.e. 66 mHz:74 mHz - about 89%), 
the color correction equipment can perform the required 
timing operation, which allows the equipment to process 
an input video signal propagating a 74 mHz in real time. 
[0030] Each color correction block has two video in- 
puts, a first input connected to the output from the pre- 
vious block r and a second input connected to an unproc- 
essed video signal. Each block includes a video multi- 
plexer that operates as a switch allowing the colorist to 
select either the output of a previous block or the un- 
processed video input signal as the input for a given 
block. Delay buffers are positioned in the unprocessed 
video signal between each color correction block to syn- 
chronize the unprocessed video signal with the output 
video signal from the preceding color correction block. 
Thus, the color correction blocks may be applied sepa- 
rately (i.e., in parallel) or in tandem (i.e., in series) for 
each scene processed. In addition, a particular colorist 
may select the number of blocks in the system in ac- 
cordance with that colorisfs needs. 
[0031] To define color correction for a scene of a video 
image, a colorist first defines a scene and selects one 
of the frames of the scene as a target frame for manual 
manipulation. The image processing system includes a 
user interface that allows the colorist to select hue sec- 
tors for isolation by placing a cursor on a desired location 
of the target frame. The user interface displays the hue, 
saturation, and luminance of the sample of pixels under 
the cursor. The user interface also displays qualification 
curves about these hue, saturation, and luminance val- 
ues. The colorist may then adjust the qualification 
curves to define the shape and size of the hue sector 
that defines the color correction region . The qualification 
curves defined about the hue, saturation, and luminance 
values displayed on the user interface determine both 
a hue sector to receive color correction and a qualifica- 
tion curve establishing the percent of color correction to 
be applied over the hue sector. 

[0032] The user interface also allows the colorist to 
select corrected color values for each hue sector. Each 



color correction block blends the input color values with 
the corrected color values on a pixel-by-pixel basis 
through application of an alpha qualifier in an alpha mix- 
er. The alpha qualifier reflects the product of the hue, 

5 saturation, and luminance qualifiers for a particular pix- 
el's color value, where each qualifier is a point on a cor- 
responding qualification curve. The final alpha qualifier 
may also reflect an additional alpha filter qualifier that 
further modifies the product of the hue/saturation, and 

10 luminance qualifiers. The final alpha qualification curve 
typically includes a full color correction zone around the 
hue, saturation, and luminance values of the sample of 
pixels under the cursor. In this full color correction zone, 
the color value of the output signal is equal to the cor- 

15 rected color value. That is, the input color value is are 
completely replaced by the corrected color value for in- 
put color values that fall within the full color correction 
zone. 

[0033] The final alpha qualification curve also includ- 

20 ed transition zones within the hue sector but outside of 
the full color correction zone. In the transition zones, the 
alpha mixer uses the alpha qualification curve to blend 
the input color values with the corrected color values set 
by the colorist. For a given pixel, the output color value 

25 is computed as a weighted sum of the input color value 
and the corrected color value, where the alpha qualifier 
defines the weighting coefficients. The alpha mixer thus 
permits smoothing of the color correction, or a gradual 
reduction in the amount of color correction applied at the 

30 edges of the hue sector. As a result, the output color 
values typically include increasing amounts of the input 
color values as the hue-domain color parameters (hue, 
saturation, and luminance) diverge from the hue-do- 
main values of the sample of pixels under the cursor. 

35 Because the colorist may define the hue, saturation, and 
luminance qualifiers independently, the blend at the 
edges of the hue sector may be based on hue, satura- 
tion or luminance, or a combination of these parameters. 
As noted previously, an additional alpha filter may fur- 

40 ther shape the final alpha qualifier applied by the alpha 
mixer 

[0034] Each color correction block allows the colorist 
to define qualifiers and color correction values for up to 
sixteen non-overlapping hue sectors, which are also re- 

45 ferred to as color correction channels. As an additional 
option, the user interface allows the colorist to define 
one or more geometric constraints for each color cor- 
rection channel. For example, a geometric constraint 
may be a window that restricts the application of color 

50 correction for the corresponding hue sector to the por- 
tion of the frame inside : or alternatively outside, the win- 
dow. Geometric constraints allow the colorist to limit the 
automatic application of color correction to a defined 
portion of the frame area, over the several frames of a 

55 scene. Geometric constraints therefore increase the 
system's ability to isolate discrete objects in a video 
scene for color correction. 

[0035] Once the colorist is satisfied with the color cor- 
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rection defined for the target frame, the color correction 
system automatically applies the color correction pa- 
rameters defined for the target frame to the other frames 
of the scene. That is, the color correction system iso- 
lates color correction regions in each frame of the scene 
using the qualification curves defined for the target 
frame. The system also mixes the corrected color values 
defined for the target frame with the input color values 
of each frame of the scene using the qualification curves 
defined for the target frame. Specifically, the color cor- 
rection system applies the corrected color values to the 
isolated regions by mixing the input color values with the 
corrected color values on a pixel-by-pixel basis through 
application of an alpha qualifier in the alpha mixer. The 
steps are repeated for other scenes in the film or video 
tape, often with different correction settings stored for 
different scenes. This process may be repeated as 
needed to create a color corrected master film or video 
tape that reflects multiple color adjustments to multiple 
frames in multiple scenes. 

[0036] Although the disclosed embodiment of the in- 
vention is described in the context of a scene-by-scene 
digital color correction system, those skilled in the art 
will understand that the invention is not limited to video 
colorcorrection, but may be equally useful in othertypes 
of color image processing in which there is a need for 
region isolation, such as image compression, image fil- 
tering, noise suppression, animation, and many other 
image processing techniques. The invention is particu- 
larly useful in an image processing application in which 
there is a need to define a first region of an image for 
receiving one type of data processing, and another re- 
gion for receiving a different type of data processing. 
The disclosed embodiment, however, is specifically tai- 
lored to operate as a scene-by-scene color correction 
system in which several frames defining a scene of a 
video program may receive automatic color corrections 
defined for a selected target frame of the scene. Thus, 
the disclosed embodiment is particularly useful as a 
color correction or color enhancement system for video 
programs, including motion pictures, television pro- 
grams, video-taped recordings, and the like. 
[0037] Certain methods associated with the disclosed 
embodiment are presented in terms of exemplary dis- 
play images, algorithms, and symbolic representations 
of operations of data within the computer's memory. As 
known to those skilled in the programming arts, these 
are typical techniques used by those skilled in the art of 
computer programming and computer construction to 
convey teachings and discoveries to others skilled in the 
art. The description of the methods involved in the 
present invention include numerous details, such as 
computer display system elements, aspects of compu- 
ter program user interfaces, display screens and for- 
mats, sample data, and so forth, in order to provide an 
understanding of the invention. Nevertheless, those 
skilled in the art will understand that the invention may 
be practiced without these specific details. In addition, 



well-known circuits, programming methodologies, and 
structures utilized in the present invention may not be 
described in detail to avoid obscuring the inventive as- 
pects of the disclosed embodiments. 

5 [0038] For purposes of this discussion, an "algorithm" 
is generally a sequence of computer-executed steps 
leading to a desired result. These steps require physical 
manipulations of physical quantities. Usually, though not 
necessarily, these quantities take the form of electrical, 

10 magnetic, or optical signals that may be stored, trans- 
ferred, combined, compared, or otherwise manipulated. 
Those skilled in the art typically refer to these signals as 
bits, values, elements, symbols, characters, images, 
terms, numbers, or the like. It must be understood that 

15 no involvement of a human operator is necessary or 
even desirable in many aspects of the present invention, 
since the operations described (except those specifical- 
ly identified as originating with the operator, user, or 
colorist) are machine operations performed in conjunc- 

20 tion with a human operator that interacts with the com- 
puter. 

[0039] Furthermore, it should be kept in mind that 
there is a distinction between the methods, steps, or op- 
erations carried out by a computer, and the method of 

25 computation itself. The present invention does not in- 
volve methods of computation. The invention rather re- 
lates to methods, steps, or operations for a computer 
processing electrical or other physical signals, generat- 
ing desired physical signals, and displaying results and 

30 user interfaces. As shown in FIG. 1 , the present inven- 
tion relates to a system 10 for performing these opera- 
tions. 

[0040] It should be further understood that the opera- 
tions described in this specification are not related or 

35 limited to any particular computer or apparatus. Rather, 
various types of general purpose or specialized ma- 
chines may be used with programs constructed in ac- 
cordance with invention. These machines may be gen- 
eral purpose digital computers, application-specific in- 

40 tegrated circuits : parallel-processing hardware, neural 
networks, or othertypes computing devices. Neverthe- 
less, it is considered advantageous to construct a spe- 
cialized apparatus to perform the operations associated 
with the invention using parallel-processing circuitry, 

45 such as programmable logic units (PLUs), field-pro- 
grammable gate arrays (FPGAs), or othertypes of re- 
programmable parallel-processing hardware. 

Image Processing System 

50 

[0041 ] Referring now to the drawings, in which like nu- 
merals indicate like elements or steps throughout the 
several figures, FIG. 1 illustrates an image processing 
system 10 constructed in accordance with an illustrative 
55 embodiment of the present invention. The image 
processing system 10 is suitable for processing video 
data by carrying out primary color manipulation using 
secondary color correction region isolation. The image 
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processing system 10 includes several basic compo- 
nents: a computer-based workstation 12 including mon- 
itor 15 and keyboard 20, a colorist's control panel 25, a 
digital color correction system 30, a source of video im- 
ages 32, a video display 40, and a computer storage 
medium for processed images 42. The workstation 12 
preferably includes pointing devices, such as a mouse 
18 and/or graphics tablet 19. The workstation 12 is cou- 
pled for data communications via a network connection 
22, preferably a high speed network such as an Ether- 
net, to the color correction system control panel 25 and 
the color correction system 30. 
[0042] The source of images 32 may be an analog 
source, such as a telecine or video tape player, or the 
source may be a digital video storage medium. The 
source is connected to an input circuit board 31 in the 
color correction system 30, which converts analog input 
signals to digital signals if necessary. The image signals 
Internal to the color correction system 30 are digital, and 
are stored and manipulated in a digital video format, 
such as the International Radio Consultant Committee 
(CCIR) promulgated Recommendation No. 601-1 
(1986), which is incorporated into this specif ication_by_ 
reference. The standard signals defined in Recommen- 
dation 601 -1 essentially consist of a luminance signal Y 
and two color difference signals (R-Y) and (B-Y). It Is 
well known that, since the luminance signal contains in- 
formation on levels of red, green . and blue (R,G,B), the 
three standard 601-1 signals can be used to reproduce 
the R f G,B levels for any given set of samples. 
The color correction system 30 converts the 601 -1 sig- 
nals to corresponding signals in the primary color do- 
main (i.e., digital R,G,B signals). The color correction 
system 30 then manipulates the resultant signals in the 
primary color domain. The corrected or modified signal 
can then be passed through a signal matrix to convert 
the modified digital signal to an analog format suitable 
for storage on a medium, such as video tape, that may, 
in turn, be used to drive an analog display device, such 
as a television or other video display. The color correc- 
tion system 30 also includes an output circuit board 35 
that typically converts the digital signals within the color 
correction system into analog form for immediate dis- 
play to the operator of the system. Thus, the output cir- 
cuit 35 is coupled via a video signal cable 37 to a video 
monitor 40 and to a utilization device 42, such as a video 
tape recorder or other image storage device. Alterna- 
tively, the corrected digital signals can be stored in dig- 
ital form in a digital data storage device coupled to the 
system. 

[0043] Because the image sou rce 32 is located down- 
stream from the SGI workstation 12, and the workstation 
uses color data to isolate color correction regions, the 
image processing system 10 includes mechanism to 
provide an input color video signal to the workstation. 
This mechanism is a splitter 104 that divides the signal 
of the output video cable 37 to provide the input color 
video signal for the workstation 12. One output of the 



splitter 104 is connected via an input video signal cable 
102 to the SGI workstation 12. The other output of the 
splitter 104 is connected via the output video cable 37 
to the video monitor 40 and utilization device 42. 

5 The preferred graphics tablet 19, which is particularly 
useful for certain manipulations of geometric con- 
straints, is a Wacom ArtPad™ or ArtZ™ graphics tablet 
manufactured by Wacom Technology Corporation, Van- 
couver, WA 98661. 

10 [0044] The preferred workstation 12 is an INDY™ 
model workstation manufactured by Silicon Graphics, 
Inc. (SGI). The preferred workstation includes an R4600 
RISC processor operating at least at 133 MHz, internal 
24-bit XL color graphics (with R,G,B single- or double- 
ts buffer color indices), a digital video input port capable 
of storing 640 x 480 x 30 fps to memory, 2 serial and 1 
parallel data port, an Ethernet network port, a fast SCSI- 
2 data port, at least 32 megabytes of RAM, and a 1 .0 or 
larger Gb hard disk. The workstation 12 also includes 

20 an on-board display driver circuit, namely, the 24-bit XL 
color graphics circuitry with R,G,B double-buffer color 
indices, which provides video signals to at least one 
workstation monitor 15. A separate, second video mon- 
itor displays a target image, typically a full-screen rep- 

25 resentation of the video image being processed by the 
system. Optionally, a different SGI model workstation 
may be employed, for example one including a video 
adapter card that drives multiple monitors. If a worksta- 
tion model is employed that supports multiple monitors, 

30 the second monitor may also be connected to the work- 
station. Further details of the preferred workstation are 
available in the literature supplied by the manufacturer. 
[0045] The preferred color correction system 30 is a 
Da Vinci RENAISSANCE 8:8:8™ digital color corrector 

35 manufactured by Da Vinci Systems, Inc., Ft. Lauderd- 
ale, Florida (assignee of the present application). This 
color correction system 30, which is coupled to the con- 
trol panel 25, includes various circuit board components 
that carry out digital video signal processing. The pre- 

40 ferred RENAISSANCE 8:8:8™ control panel 25 in- 
cludes three separate panels, each having a combina- 
tion of dedicated keys, soft keys, and soft knobs that 
provide many simultaneous functions utilized in the 
color correction process. A dedicated keyboard 50 con- 

45 tains both dedicated and soft keys for system feature 
control, display/list management, source and destina- 
tion motion control, editing, special effects features, and 
the like. This keyboard also contains a slide-out alpha- 
numeric keyboard (not shown) to enter names, com- 

50 ments, and other important session information. Includ- 
ed on this control panel is a "User Definable Windows" 
(UDW) button or key 51 that is utilized to activate the 
certain function, such as those associated with defining 
geometric constraints. 

55 [0046] The preferred RENAISSANCE 8:8:8™ system 
includes software that is operative for controlling the im- 
age source and destination, and reading and storing 
frame numbers in association with user-defined scene 
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identifiers or names. Thus, the system can display 
scene lists that facilitate the user in associating scene 
identifiers or names with selected frames from the 
scenes. In addition, the system includes control circuitry 
responsive to a house clock for triggering frame buffers 
and other video signal control devices to synchronize 
multiple video signals simultaneously under the sys- 
tem's control. This allows multiple versions of the same 
scene, such as modified and unmodified versions, to be 
synchronized for simultaneous display or combined 
processing. 

[0047] The control panel 25 also includes a joyball 
panel 53, which includes knobs, a trackball 54 for ma- 
nipulating a cursor, keys for source setup, primary color 
balance, color grabbing, trackball control, and other ef- 
fects control. When the windows feature of the present 
invention is activated, movement of the trackball 54 
causes generation of a cursor 60 that is displayed on 
the video monitor 40. The cursor 60 is utilized to manip- 
ulate and select portions of video images displayed on 
the monitor, as will be described. 
[0048] Finally, the control panel 25 includes a soft 
panel 57, which contains knobs and softkeys for dedi- 
cated 6-color secondary enhancement, special effects 
control, dedicated memory keys, and otherf unctions not 
forming a part of the present invention. In accordance 
with the invention, the control panel 25 includes features 
to facilitate primary color correction and secondary color 
correction region isolation, the definition of geometric 
constraints, and the definition of multiple sets of primary 
color correction parameters (e.g., one for regions inside 
a geometric constraint and one for regions outside the 
constraint) . These and other parameters may be set and 
stored in memory by manipulation of controls on the soft 
panel 57. 

[0049] The color correction system 30 includes sev- 
eral circuit boards that perform the functions required to 
carry out color correction and other types of image 
processing in digital color images, and to control input 
from the image source 32 and utilization or storage in 
the utilization device 42. These circuits include a real 
time controller (RTC) circuit 70 with network interface 
that provides a data communication network interface 
to the data communication network 22 coupling the color 
correction system 30 to the workstation 1 2 and color cor- 
rection system control panel 25. 
[0050] The real time controller (RTC) 70 includes a 
dedicated microprocessor for handling real time events. 
Real time events occur at the rate of one every 30 mil- 
lisecond (ms), so as to maintain an event rate corre- 
sponding to at least a 33 frames per second (fps) display 
rate. The RTC 70 receives an events via the data com- 
munications network 22 to control various functions of 
the system, for example providing a set of color correc- 
tions for each frame of a specified. In particular, the 
events list includes a data structure that identifies a par- 
ticular frame number and scene number, information 
that specifies one or more sets of color correction pa- 



rameters for the selected frame, hue-domain informa- 
tion for isolating color correction regions, information de- 
fining one or more geometric constraints for the frame, 
information identifying which color corrections are asso- 
5 ciated with which hue-domain region and geometric 
constraint, and so forth. 

[0051] The events list is a data structure having frame 
numbers as a time reference. The data structure in- 
cludes a sequence of data blocks in which each block 

10 js related to a beginning frame number and an ending 
frame number. Each block is called a "scene," which typ- 
ically corresponds to a continuous segment of film or 
video shots. Each block also contains a set of color cor- 
rection parameters, hue-domain isolation parameters, 

*5 and geometric constraint information for the frame. 
[0052] A user definable windows (UDW) generator 
board 75 includes a dedicated high speed microproces- 
sor that converts vectorized user-defined windows into 
a video matte or geometric constraint on a real time ba- 

20 sis at the real time event rate. The RTC circuit 70 pro- 
vides the vector representations of the windows to the 
UDW board 75 at the real time event rate. 
[0053] A key frame buffer (KFB) circuit board 78 
stores a video image from the image source 32, or from 

25 any other digital storage system, in a high speed mem- 
ory. This image is then utilized in to derive a vector rep- 
resentation of any user defined windows. Keys for vec- 
torizing may be obtained directly from the image source 
32, or may be retrieved from prestored images stored 

30 in the memory of the workstation 12. 

[0054] Color correction equipment 80 carries out dig- 
ital color correction and other video parameter adjust- 
ment functions associated with the color correction sys- 
tem 30. The block in FIG. 1 identified as the color cor- 

35 rection equipment 80, which includes all of the remain- 
ing functions of the preferred RENAISSANCE 8:8:8™ 
digital color correction system 30, is described in greater 
detail with reference to FIGS. 2-14. Briefly, the color cor- 
rection equipment 80 receives color correction param- 

40 eters that define the coefficients of a transformation ma- 
trix or T-matrix" that is applied to an input R,G,B, signal 
to produce a color corrected output R,G : B signal. The 
input R : G,B signal and the output R,G,B signal from the 
T-matrix process are combined within an "alpha mixer" 

^5 that restricts that application of the color corrections de- 
fined in the T-matrix to color correction regions defined 
in the hue-domain through hue, saturation, and lumi- 
nance qualification curves. The hue, saturation, and lu- 
minance qualification curves are non-additively com- 

50 bined to produce a total qualification curve. An addition- 
al alpha filter may further modify the total qualification 
curve to produce the final alpha qualification curve that 
is applied by the alpha mixer. For a given pixel, the out- 
put color value is computed as a weighted sum of the 

55 input color value and the corrected color value, where 
the alpha qualifier (i.e., a point on the alpha qualification 
curve) defines the weighting coefficients. 
[0055] Thus, the preferred color correction system 30 
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is operative to receive video images from an image 
source such as telecine 32, digitize the video images, 
capture a digitized image in the KFB board 78, display 
the captured image on the video monitor 40, receive 
color corrections entered by the colorist or operator via 
the control panel 25, impose these color corrections up- 
on video images as they are transferred through the sys- 
tem, and provided corrected output signals via the out- 
put board 35. These output signals are employed or 
stored in a utilization device 42 and/or simultaneously 
displayed on the video monitor 40. 
[0056] In carrying out the operations for applying im- 
age processing to an input video signal, a system oper- 
ator or colorist interacts with the system through the con- 
trol panel 25, and via the user interface elements of the 
workstation 12, including the keyboard 20, mouse 18, 
graphics tablet 19, and workstation monitor 15. In par- 
ticular, the workstation 12 executes computer programs 
for carrying out the computer-implemented methods de- 
scribed in this specification, and generates displays on 
the monitor 15 windows containing user interface ele- 
ments, such as informational displays, control buttons, 
slider bars, data entry fields, image display areas, eta 
[0057] The reader should be careful here to distin- 
guish user- interface "windows" displayed on the monitor 
15 generated from execution of computer software for 
carrying out the invention, from the "user-defined win- 
dows" for applying image processing. During system 
use, the workstation monitor 15 typically displays one 
or more user-interface windows for carrying out the 
methods as described herein. One such user-interface 
window is a control window 44, which includes a display 
of various buttons and slider controls for manipulation 
of the key image. Activation of certain commands caus- 
es the display of other, secondary windows. One of the 
secondary windows is a View/Edit Window 46, shown 
overlapping the control window 44 in FIG. 1 . The View/ 
Edit Window 46 is used to display a captured image in 
greater detail, andtoprovide certain additional user con- 
trols specifically associated with editing of the image. 
[0058] It should be noted that the video image dis- 
played in the video monitor 40 can be displayed within 
a defined View/Edit window 46 if desired. Since the im- 
ages provided from the image source are digitized for 
processing in the preferred .RENAISSANCE 8:8:8™ 
processing system, the system can readily generate 
digitized video images and provide them in an appropri- 
ate format, such as Apple Computer's Quicktime™ vid- 
eo format, MPEG I or II, or other digital video format via 
the network 22 to the workstation 12. Accordingly, any 
of the displays that are described as being shown on the 
video monitor 40 can alternatively be displayed on the 
workstation monitor 15. Furthermore, any of the func- 
tions or displays described in conjunction with the video 
monitor 40 can also be supplied in a window in the work- 
station display 15. 

[0059] It is to be expected that, as more and more im- 
age processing of film and video is conducted in the dig- 



ital domain as opposed to the traditional analog domain, 
methods of the present invention will be utilized in strictly 
digital systems for image storage, retrieval, manipula- 
tion, display, and output. In the disclosed embodiment, 
5 however, the results of image processing are viewed on 
a conventional analog video monitor 40, while the user 
interface and controls aspects of the image processing 
and the invention are carried out through the workstation 
12 and its associated monitor 15. 

10 

Overview of Color correction Methodology 

[0060] FIG. 2 is a schematic block diagram illustrating 
the use of the image processing system 1 0 for imple- 

15 menting color correction region isolation using second- 
ary color parameters. FIG. 2 illustrates the system 10 
after the colorist has selected an input image including 
a scene from the image source 32 to receive color cor- 
rection. The colorist has also selected a particular target 

20 frame of the scene to receive manual color correction. 
For the example shown in FIG. 2, the frames 110a-n 
define the scene and the frame 1 1 0a is the target frame 
that the colorist has selected for manual adjustment. As 
the colorist adjusts the target frame 110a, this frame is 

25 typically displayed on the video monitor 40 while a color 
correction user interface 120 is displayed on the work- 
station display 15. The color correction user interface 
120 is described in greater detail with reference to FIGS. 
5 and 6A-B. 

30 The colorist operates the various controls of the image 
processing system 1 0, represented by the control panel 
25, to manually enter color corrections for the selected 
frame 110a. This process includes isolating color cor- 
rection regions within the input image, defining color cor- 

35 rection parameters that establish the corrected colors 
for the color correction regions, and defining an alpha 
qualification curve, which is a smoothing function for 
blending the input color values with the corrected colors 
selected by the colorist. Once the colorist is satisfied 

40 with the color corrections for the selected frame 110a, 
the color correction equipment 80 automatically applies 
these color corrections to the remaining scenes of the 
frame without the colorist having to manually adjust 
each frame of the scene. 

^5 As noted previously, automatically isolating a color cor- 
rection region presents a difficult technical problem be- 
cause the region of interest typically changes in size, 
shape, location, and/or geometry over the frames in a 
scene. For example, in the example shown in FIG. 2, a 

50 bottle 100 is lifted, tilted, moved toward a glass 105, 
turned toward the viewer, and then further tilted to pour 
the contents of the bottle into a glass. In this scene, the 
geometry of the bottle 100 changes from a side view of 
the bottle to a top view of the bottle. The bottle also 

55 changes in location and size over the several frames of 
the scene. The color correction equipment 80 imple- 
ments an improved method for automatic color correc- 
tion region isolation over the frames of the scene so that 
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the color corrections defined by the colorist for the se- 
lected frame 1 1 0a are properly applied to the remaining 
scenes of the frame. 

The image processing system 10 typically receives im- 
age data from the image source 32 in a primary color 
domain, such as 24-bit R,G,B or in the format specified 
by CCIR Recommendation No. 601-1. Isolating color 
correction regions by manipulating the primary color do- 
main pararneters : however, often produces unsatisfac- 
tory results. For this reason, color correction systems 
may apply color corrections by transforming the source 
image into a secondary color domain, applying color 
correction parameters in the secondary color domain, 
and then transforming the corrected image back to the 
primary color domain. In particular, the R,G,B source im- 
age may be linearly transformed into the hue domain, 
which includes hue, saturation, and luminance param- 
eters. The relationship between color values in the pri- 
mary color domain and in the hue domain is described 
in greater detail with reference to FIG. 3. 
[0061] But applying color corrections in the hue do- 
main is computationally expensive and often introduces 
color artifacts into the color corrected image. The color 
correction equipment 80 within the system 10 avoids 
this problem by isolating color correction regions using . 
qualification curves defined the hue domain. The color 
correction parameters for these regions, however, are 
defined in the primary color domain. The qualification 
curves also serve as smoothing functions for blending 
the input color values with the color correction parame- 
ters selected by the colorist. That is, the qualification 
curves are non-additively combined, optionally with an 
additional alpha filter, to produce an alpha qualification 
curve that the color correction equipment 80 uses to 
blend the input image with the color correction parame- 
ters selected by the colorist. The color correction equip- 
ment 80 performs the blending operation, which is re- 
ferred to as "alpha mixing," using the input color values 
and the corrected color values in the primary color do- 
main. 

To define a color correction region, the colorist places 
the cursor 60 on a sample of the selected frame 110a. 
This sample may be a single pixel or a number of pixels 
that the color correction equipment 80 averages to ob- 
tain a representative set of color values. In either case, 
the sample may be represented by a set of color param- 
eters preferably expressed by the hue domain parame- 
ters hue, luminance, and saturation . These color param- 
eters for the sample are then displayed within the user 
interface 120 on the workstation display 15. The user 
interface 120 allows the colorist to define a hue qualifi- 
cation curve about the hue of the sample, a saturation 
qualification curve about the saturation of the sample, 
and a luminance qualification about the luminance of the 
sample, as represented by the three trapezoidal curves 
shown on the user interface 120. 
[0062] The colorist uses the control panel 25 to select 
corrected color values for the sample and to manipulate 



the qualification curves as desired. The colorist may al- 
so use the control panel 25 to define an additional alpha 
filter and a geometric constraint for the selected sample. 
The result of the color corrections entered by the colorist 

5 for the target frame 110a is typically displayed on the 
video display 40. The colorist may then repeat the proc- 
ess for another sample of the target frame 110a until all 
of the desired color correction have been entered, or un- 
til all of the color correction channels available in the 

10 system 10 are occupied. Once the colorist is satisfied 
with the color corrections for the target frame 1 10a, the 
color correction equipment 80 automatically applies 
these color corrections to the remaining scenes of the 
frame. The colorist repeats this process as needed to 

'5 create a color corrected master film or video tape that 
reflects multiple color adjustments to multiple frames in 
multiple scenes. 

Relationship Between Primary and Secondary 
20 Color Domains 

[0063] FIG. 3 (prior art) is a diagram 300 illustrating 
the relationship between the additive primary colors, a 
color wheel, and a color bar. Video display and projec- 
ts tion systems typically operate using the additive primary 
colors, red, green, and blue. For example, a typical 
24-bit color value in the primary color domain includes 
an 8-bit red parameter, an 8-bit green parameter, and 
an 8-bit blue parameter. These three color parameters 
30 may be combined in varying intensities to produce 256 
different colors values. When all three primary colors are 
combined in full intensity, they produce the whitest color 
that the device displaying the signal can produce. The 
absence of all three primary colors produces the black- 
35 est color that the device displaying the signal can pro- 
duce. Between these two extremes, the three primary 
colors in equal proportions produce various shades of 
gray, depending on the intensity of the primary color 
constituents. Non-equal proportions of the primary 
40 colors produce all of the other colors that the device dis- 
playing the signal can produce. For example, red and 
green may be combined in equal proportions to produce 
yellow, green and blue may be combined in equal pro- 
portions to produce cyan, and blue and red may be com- 
45 bined in equal proportions to produce magenta, and so 
forth. 

[0064] The colors of the visible spectrum may be plot- 
ted in increasing wavelength (decreasing frequency) on 
a color circle 302, which begins in the magenta region, 

50 starting at about 400 millimicrons wavelength at zero de- 
grees, and continues around the circle to the red region, 
ending at about 700 millimicrons wavelength at 360 de- 
grees. The hue of a particular color represents the rel- 
ative proportions of the primary-color constituent. As 

55 such, the hue may be represented by a particular angle 
304 on the color circle 302. Just as may be grays may 
be displayed in varying intensities, so may every other 
hue that a display device can produce. The outer edge 
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of the color circle 302 represents the highest-intensity 
value of each hue, and the center of the circle represents 
black or the absence of all three primary-color constitu- 
ents. The relative intensity, or saturation, of a color value 
having a particular hue may therefore be represented 5 
as the length of a vector 306 displayed and the angle 
304 representing the hue. 

[0065] In some cases, it is more convenient to "roll 
out" the color circle 304 to produce a color bar 308, also 
known as a rainbow. In this color bar, the zero degree 10 
point of the color circle 302 is typically displayed at the 
left edge of the color bar, and the 360 degree point is 
typically displayed at the right edge of the color bar. 
Thus, the color bar 308 extends horizontally, from left to 
right, beginning in the magenta region at about 400 mil- 15 
limicrons wavelength, and ending in the red region at 
about 700 millimicrons wavelength. The hue 304 of a 
color value may therefore be represented as a horizon- 
tal position along the color bar 308. In addition, the 
height of the color bar 308 represents the saturation 20 
scale, with black at the bottom of the bar and the full- 
intensity values at the top of the bar. This allows the sat- 
uration 306 of a color value to be represented by the 
height of a vector displayed at the horizontal position 
representing the hue. 25 
[0066] FIG. 4 (prior art) is a diagram illustrating the 
relationship between color values in the primary color 
domain and in the hue domain. In addition to the hue 
304 (represented as "H") and the saturation 306 (repre- 
sented as "S"), an R,G f B color value in theprimary color 30 
domain also reflects a third parameter, luminance (rep- 
resented as "Y"), which along with the hue and the sat- 
uration fully represent the color value in the hue domain. 
The concept of luminance involves the relative bright- 
ness of the various colors of the visible spectrum as per- 35 
ceived by the human eye. This phenomenon is repre- 
sented by the luminance curve 320 shown in FIG. 4. Al- 
though the luminance curve 320 shown in FIG. 4 is not 
intended to be precisely accurate, it is indicative of the 
approximate shape of the actual luminance curve, which 40 
can be found in any number of generally available text 
books that describe color correction technology. 
[0067] An R,G,B color value 322 in the primary color 
domain may be transformed into an H,S,Y color value 
342 in the hue domain through the application of the lin- 45 
ear transformation matrix 326. That is, the H,S,Y color 
parameters can be computed as weighted sums of the 
R,G,B color parameters, and vice versa. Those skilled 
in the color correction art are familiar computing tech- 
niques for transforming R,G,B color values to H,S,Y 50 
color values, and then converting the H,S,Y color values 
back to R,G,B, color values. Unlike prior image process- 
ing systems, however, the image processing system 80 
takes advantage of this fluidity of color value represen- 
tation to implement color correction region isolation in 55 
the hue domain, while mixing input color values and 
color in the primary color domain. 



Color Correction User Interface 

[0068] FIG. 5 is a diagram illustrating the user inter- 
face 120 of FIG. 2 in greater detail. In the example illus- 
trated in FIG. 2, the video monitor 40 displays the target 
image, which is the selected frame 110a of the scene 
that the colorist is in the process of color correcting. The 
colorist defines a color correction region by first placing 
the cursor 60 over a sample of the target image. This 
selection causes the workstation monitor 15 to display 
the user interface 120, which includes a hue isolation 
user interface 504, a saturation user interface 506, and 
a luminance user interface 508. 
[0069] The hue isolation user interface 504 displays 
the selected hue 510 (i.e., the hue of the sample) within 
a hue qualification curve 51 2. The selected hue and the 
hue qualification curve 512 are displayed within a color 
bar 514 of the type described with reference to FIG. 3. 
The color bar 514 preferably shows the various colors 
represented by the bar, and for this reason is often re- 
ferred to as the rainbow. The height of the color bar rep- 
resents the percent qualification with respect to the hue 
qualification curve 512, The workstation monitor 15 typ- 
ically illustrates only a portion of the color bar 514, and 
the colorist may scroll the display of the workstation 
monitor 1 5 to the left or right to cause a different portion 
of the color bar to appear on the monitor. 
[0070] The hue qualification curve 512 is generally 
trapezoidal in shape and defines, at least in part, a hue 
sector in which color correction parameters will be ap- 
plied. The colorist selects a corrected color for the hue 
sector and may alter both the shape the of the hue qual- 
ification curve 512 and its position along the color bar 
514. The corrected color value selected by the colorist 
corresponds to the color correction parameters. Specif- 
ically, the color correction parameters are the coeffi- 
cients of a transformation matrix, referred to as the T- 
matrix," that converts the input color value to the cor- 
rected color value. That is, the input color value multi- 
plied by the "T" matrix" produces the corrected color val- 
ue selected by the colorist. 

[0071 ] The saturation isolation user interface 506 dis- 
plays the selected saturation 516 (i.e., the saturation of 
the sample) within a saturation qualification curve 518. 
The selected saturation 516 and the saturation qualifi- 
cation curve 518 are displayed within a saturation bar 
520 having the percent saturation increasing from left to 
right in the horizontal direction, and the percent qualifi- 
cation increasing from bottom to top in the vertical di- 
rection. The saturation qualification curve 518 is gener- 
ally trapezoidal in shape and also contributes to the def- 
inition of the hue sector in which color correction may 
be applied. The colorist may alter both the shape the of 
the saturation qualification curve 518 and its position 
along the saturation bar 520. 

[0072] In similar fashion, the luminance isolation user 
interface 508 displays the selected luminance 522 (i.e., 
the luminance of the sample) within a luminance quali- 
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f ication curve 524. The selected luminance 522 and the 
luminance qualification curve 524 are displayed within 
a luminance bar 526 having the percent luminance in- 
creasing from left to right in the horizontal direction, and 
the percent qualification increasing from bottom to top 5 
in the vertical direction. The luminance qualification 
curve 524 is generally trapezoidal in shape and also 
contributes to the definition of the hue sector in which 
color correction may be applied. The colorist may alter 
both the shape the of the luminance qualification curve 10 
524 and its position along the luminance bar 526. 
[0073] Because the workstation monitor 15 displays 
the hue spectrum within the color bar 51 4 while the video 
monitor 40 displays the target image, the combined dis- 
plays assist the colorist in selecting a hue sector that 15 
captures a particular object included in the target image 
while avoiding other objects that may have similar, but 
perceptibly different, hues. In addition, the colorist may 
preview the effect of the selected color correction on the 
target frame 110a displayed on the video monitor 40. In 20 
other words, the colorist may toggle the display shown 
on the video monitor 40 between the unprocessed input 
image and a processed image for the scene 110a to 
gauge the effect of the selected color correction. 
[0074] As mentioned previously, color correction re- 25 
gion isolation is performed in the hue domain, while the 
color correction itself is performed in the primary-color 
domain. A particular pixel having an R t G,B color value 
has a corresponding H,S,Y color value. The hue param- 
eter (H) has a non-zero hue qualifier (i.e., qualification 30 
percent) (oc h ) if it is between H1 and H2 as shown in the 
hue isolation user interface 504. The hue qualifier (a^ 
may typically be determined through reference to a look- 
up table that contains the hue qualification curve 512. 
Similarly, the saturation parameter (S) has a non-zero 35 
saturation qualifier (a s ) if it is between S1 and S2 as 
shown in the saturation isolation user interface 506. The 
saturation qualifier (0^) may also be determined through 
reference to a look-up table that contains the saturation 
qualification curve 518. And the luminance parameter 40 
(Y) has a non-zero luminance qualifier (Oy) if it is be- 
tween Y1 and Y2 as shown in the luminance isolation 
user interface 508. The luminance qualifier (<Xy) may al- 
so be determined through reference to a look-up table 
that contains the luminance qualification curve 524. *s 
[0075] The hue qualification percent or qualifier (a h ), 
the saturation qualifier (a s ), and the luminance qualifier 
(ocy) may by non-additively combined, for example by 
multiplying the three percentages together, to obtain a 
total alpha qualifier (04) for the particular pixel. The total so 
alpha qualifier (04) may be further modified by an option- 
al alpha filter qualification percentage to produce a final 
alpha qualifier (a) for the pixel. The final alpha qualifier 
(a) is a value between zero and 1 00 percent that deter- 
mines the relative portions of the input color value (R in , 55 
Gj^BjJ and the corrected color value (R'.G'.B 1 ) in the 
output pixel, where (a) represents the relative contribu- 
tion of the corrected color value and (1-a) represents 



the relative contribution of the input color value. For ex- 
ample, if the final alpha qualifier (a) is equal to 75%, the 
color value of the output pixel (R 0 ut> G out> B out) ' s be de- 
termined by the following alpha-mixing formula: 

R 0Ut =.75*R' + .25*R jn 
G out =.75*G' + .25*G in 

B oul =.75*B' + .25*B jn 

[0076] Thus, the final alpha qualifier (a) is used to de- 
termine the weighting parameters for blending the input 
color parameters with the corrected color parameters by 
computing the output color parameters as weighted 
sums of the primary-color domain color parameters. 
This type of color blending, referred to as "alpha mixing, 
" permits smoothing of the color correction, or a gradual 
reduction in the amount of color correction applied at the 
edges of the hue sectors. That is, the output signal, 
which may be a blend of the input signal and the cor- 
rected color values, typically includes increasing 
amounts of the input signal as the hue-domain color pa- 
rameters diverge from the hue, saturation, and lumi- 
nance values of the sample pixels under the cursor. The 
amounts of the input signal and the corrected color val- 
ues included in the output signal is defined by the alpha 
qualifier. As the colorist may define hue, saturation, and 
luminance qualification curves independently, the color 
blend at the edges of the hue sector may be based on 
hue, saturation or luminance, or a combination of these 
parameters. An additional alpha filter may further influ- 
ence the alpha qualifier. 

[0077] As an additional option , the colorist may define 
a geometric constraint, represented by the constraint 
528 shown in FIG. 5, in addition to the hue-domain color 
correction region isolation parameters (i.e., the hue, sat- 
uration, and luminance qualification curves) described 
above. The geometric constraint allows color correction 
to be limited to a portion of each frame of the scene be- 
ing processed, where the limited portion is defined with 
respect to the geometric constraint. For the example 
shown in FIG. 5, the color correction defined for the bot- 
tle 100 may be applied to each entire frame except the 
portion within the constraint 502. This approach pre- 
vents color corrections intended to affect only the bottle 
100 from also affecting the glass 105. 
[0078] Thus, the geometric constraint 528 allows the 
colorist to limit the automatic application of color correc- 
tion to a defined portion of the frame area, over the sev- 
eral frames of a scene. Geometric constraints therefore 
increase the ability of the color correction equipment 80 
to isolate discrete objects for color correction. In partic- 
ular, the colorist may use a geometric constraint to ex- 
clude from color correction discrete objects that have a 
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similar color to a target object. In the example shown in 
FIG. 5, the bottle 100 is the target object and the geo- 
metric constraint 528 prevents a color correction intend- 
ed for the bottle 100 from also affecting the color of the 
glass 1 05, which may have a similar color. . 
[0079] FIG. 6A is a diagram illustrating a portion of the 
hue isolation user interface 504 in greater detail. The 
slope of the rising and falling edges of the hue qualifi- 
cation curve 512 defines the softness of the color cor- 
rection. The hue qualification curve 512 typically in- 
cludes a full color correction zone 604 at the center of 
the curve, and sloped rising and falling edges toward 
the outer edges of the hue sector. These sloped edges 
define transition zones 606a and 606b towards the edg- 
es of the hue sector. In the full color correction zone 604, 
the colors of the output signal are equal to the corrected 
color values (i.e., the color values of input signal are 
completely replaced by the corrected color values). For 
the transition zones 606a and 606b, the input color val- 
ues are mixed with the corrected color values in a per- 
centage equal to the hue qualifier (o^), which the per- 
cent qualification determined by reference to the hue 
qualification curve 512. 

[0080] Each of the qualification curves 512, 518, and 
524 may be shaped as shown for the representative hue 
qualification curve 512. In addition, any of the qualifica- 
tion curves 512, 518, and 524 may be set equal to the 
value "one" for its entire horizontal range to effectively 
remove qualification for the corresponding color param- 
eter. Thus, the color correction equipment 80 allows the 
colorist to use hue qualification, saturation qualification, 
or luminance qualification individually to select colorcor- 
rection regions. In addition, more then one qualification 
curve may be applied simultaneously to define a color 
correction region. 

[0081 ] The color correction equipment 80 is typically 
configured as one or more color correction blocks in 
which each block may be used to define up to sixteen 
non-overlapping hue sectors or color correction chan- 
nels. Accordingly, the hue isolation user interface 504 
typically displays a number of hue qualification curves 
5122a-n, each corresponding to one of the available 
color correction channels. The display shown on the 
workstation monitor 15 typically tracks the active chan- 
nel automatically so that the hue qualification curve cor- 
responding to a channel selected by the colorist auto- 
matically appears in the center of the workstation mon- 
itor 15. 

[0082] FIG. 6B is a diagram illustrating an alternative 
user interface 608 for operating the color correction 
equipment 80. The user interface 608 collapses the sat- 
uration user interface 506 and the luminance user inter- 
face 508 into compact visual elements that are conven- 
iently shown in conjunction with their corresponding hue 
qualification curve. The top of the user interface 608 in- 
cludes a hue qualification display 610, which is shown 
in this example as having three hue qualification curves 
612, 614, and 616 displayed against the background of 



a color bar 618 of the type described with reference to 
FIG. 3. Recall that each color correction block typically 
includes up to sixteen color correction channels. To il- 
lustrate a typical scenario by taking the hue qualification 
5 curves 612, 614, and 616 from left to right, the first hue 
qualification curve 612 may correspond to the first color 
correction channel (numbered M 1"), the second hue 
qualification curve 61 4 may correspond to the third color 
correction channel (numbered "3"), and the third hue 
10 qualification curve 616 may corresponds to the second 
color correction channel (numbered "2"). 
[0083] FIG. 6B shows the hue qualification curves out 
of order with respect to the numbering of the color cor- 
rection channels because the colorist may assign the 
15 color correction channels in any order (e.g., by assign- 
ing different channels to different samples of the target 
frame 110a), whereas the hue qualification display 610 
displays the hue qualification curves in hue order along 
the underlying color bar 618 or rainbow. The hue quali- 
20 fjcation curves are, therefore, superimposed against the 
color bar 618 to show the color of their respective hue 
sectors at a glance. The curves are clearly labeled with 
their respective color correction channel numbers so 
that the colorist can determine which controls will affect 
25 which hue qualification curve. This type of display allows 
the colorist to scroll left and right along the color bar 618 
to see the various hue qualification curves, and their as- 
sociated color correction channel numbers, in their re- 
spective positions along the color bar 618 . As noted 
30 previously, the colorist may alter the position and shape 
of each hue qualification curve individually. 
[0084] A hue-shift display 620 is located beneath the 
hue qualification display 610. The hue-shift display 620 
includes afirst color correction block622 located directly 
35 below the first hue qualification curve 612. The width of 
the first color correction block 622 is equal to the width 
of the first hue qualification curve 612, which sets the 
width of the hue sector for color correction channel 
number one. The color correction block 622 displays the 
corrected color for color correction channel number one. 
The corrected color is the output color value selected by 
the colorist for input color values that produce a final 
alpha qualifier (a) equal to 1 00 percent. With reference 
to the previously-defined alpha-mixing formula, the 
color correction block 622 displays the corrected color 
value (R',G\B') for color correction channel number one. 
[0085] Similarly, the hue-shift display 620 includes a 
second color correction block 624 located directly below 
the second hue qualification curve 61 4. The width of the 
second color correction block 624 is equal to the width 
of the second hue qualification curve 614, which sets 
the width of the hue sector for color correction channel 
number three. The color correction block 624 displays 
the corrected color value (R'.G'.B') for color correction 
channel number three. And the hue-shift display 620 in- 
cludes a third color correction block 626 located directly 
below the third hue qualification curve 61 6. The width of 
the third color correction block 626 is equal to the width 
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of the third hue qualification curve 616, which sets the 
width of the hue sector for color correction channel 
number two. As with the other blocks, the color correc- 
tion block 626 displays the corrected color value (R\G\ 
B') for color correction channel number two. 
[0086] The color correction user interface 608 also in- 
cludes a luminance display 630 on the left-half portion 
of the interface below the hue-shift display 620. The lu- 
minance display 630 includes three control windows 
632, 634, and 636. Each of these control windows may 
display a luminance qualification curve for a particular 
color correction channel, as determined by the colorist. 
For example, the colorist may assign the first luminance 
control window 632 to color correction channel number 
one, the second luminance control window 634 to color 
correction channel number two, and the third luminance 
control window 636 to color correction channel number 
three. 

[0087] With these assignments, the first luminance 
control window 632 displays a luminance qualification 
curve 638 for color correction channel number one, the 
second luminance control window 634 displays a lumi- 
nance qualification curve 640 for color correction chan- 
nel numbertwo, and the third luminance control window 
636 displays a luminance qualification curve 642 for 
color correction channel number three. The colorist may 
manipulate each of these luminance qualification curves 
individually. Note that each luminance qualification 
curve is rotated 90 degrees with respect to the lumi- 
nance qualification curve 518 shown in FIG. 5, so that 
the percent luminance increases from top to bottom in 
the vertical direction, and the percent qualification in- 
creases from left to right in the horizontal direction. 
[0088] The color correction channel numbers deter- 
mine which luminance qualification curve shown in the 
luminance display 630 corresponds to which hue qual- 
ification curve shown in the hue qualification display 
610. In fact, the luminance display 630 may show lumi- 
nance qualification curves that correspond to hue qual- 
ification curves that are not displayed within the hue 
qualification display 610, if the colorist so desires. 
[0089] The color correction user interface 608 also in- 
cludes a saturation display 650 on the right-half portion 
of the interface below the hue-shift display 620. The sat- 
uration display 650 includes three control windows 652, 
654, and 656. Each of these control windows may dis- 
play a saturation qualification curve for a particular color 
correction channel, as determined by the colorist. For 
example, the colorist may assign the first saturation con- 
trol window 652 to color correction channel number one, 
the second saturation control window 654 to color cor- 
rection channel number two, and the third saturation 
control window 656 to color correction channel number 
three. 

[0090] With these assignments, the first saturation 
control window 632 displays a saturation qualification 
curve 658 for color correction channel number one, the 
second control window 654 displays a saturation quali- 



fication curve 660 for color correction channel number 
two, and the third saturation control window 656 dis- 
plays a luminance qualification curve 662 for color cor- 
rection channel number three. The colorist may manip- 

5 ulate each of these saturation qualification curves indi- 
vidually. Like the luminance qualification curves, each 
saturation qualification curve is rotated 90 degrees with 
respect to the saturation qualification curve 524 shown 
in FIG. 5, so that the percent saturation increases from 

10 top to bottom in the vertical direction and the percent 
qualification increases from left to right in the horizontal 
direction. 

[0091] Also like the luminance display 630, the color 
correction channel numbers determine which saturation 
f5 qualification curve shown in the saturation display 650 
corresponds to which hue qualification curve shown in 
the qualification display 610. Again, the saturation dis- 
play 650 may show saturation qualification curves that 
correspond to hue qualification curves that are not dis- 
played within the qualification display 610, if the colorist 
so desires. 

[0092] To allow the colorist to ascertain all of the qual- 
ification information associated with a particular color 
correction channel from a single display, the hue quali- 
fication display 610 includes luminance- and saturation- 
related display elements that are shown in association 
with their corresponding hue qualification curves. This 
allows the colorist to view all of the qualification infor- 
mation by scrolling left and right along the color bar 618 
to display the hue qualification curves with their corre- 
sponding luminance- and saturation-related display el- 
ements for the various color correction channels. The 
colorist may then stop at any point along the color bar 
618, assign one of the luminance or saturation control 
windows to a particular color control channel, typically 
a displayed channel, and adjust the luminance or satu- 
ration qualification curves as desired. 
[0093] Specifically, the luminance qualification curves 
638, 640, and 642 each have an associated luminance 
display element (664, 666, and 668 respectively) that is 
shown in connection with the corresponding hue quali- 
fication curve (612, 616, and 614 respectively). Each lu- 
minance display element is a narrow band shown ap- 
proximately in the center of its corresponding hue qual- 
ification curve, which typically is the hue position of a 
sample of the target image selected by the colorist. For 
each luminance display element, the percent luminance 
increases in the vertical direction over the height of the 
color bar 618. Thus, the vertical extent of each lumi- 
nance display element visually represents the lumi- 
nance range covered by the corresponding luminance 
qualification curve. In addition, the percent luminance is 
preferably shown visually within the band. That is, the 
luminance display element contains the hue of the un- 
derlying color (i.e., the hue corresponding to the hori- 
zontal position of the luminance display element along 
the color bar 618) with varying luminance. In other 
words, the luminance of the hue shown within the lumi- 
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nance display element increases vertically from bottom 
to top across the luminance display element. 
[0094] In addition, the saturation qualification curves 
658, 660, and 662 each have an associated saturation 
display element (674, 676, and 678 respectively) that is 5 
shown in connection with the corresponding hue quali- 
fication curve (612, 616, and 614 respectively). Each 
saturation display element includes two horizontal lines 
across the corresponding hue qualification curve : al- 
though lines occurring at the top or bottom of the color 10 
bar 61 8 may not be shown for convenience. As the color 
bar 618 shows the rainbow in increasing saturation ver- 
tically across the color bar, the horizontal lines of satu- 
ration display elements visually represents the vertical 
extent of the saturation range covered by the corre- 15 
sponding saturation qualification curve. 
[0095] Although the luminance display 630 and the 
saturation display 650 shown in FIG. 6B depict the lu- 
minance and saturation qualification curves as trape- 
zoids, those skilled in the art will appreciate that other 20 
types of figures could equivalent^ communicate the rel- 
evant information to the colorist. For example, the dis- 
plays could showonly the slope of the rising and falling 
edges (i.e., the transition zones) of a qualification curve. 
That is, the potion of the qualification curve that is equal 25 
to one (i.e., the full qualification zone) need not be dis- 
played. This type display showing only the transition 
zones of a qualification curve would communicate all of 
the relevant information regarding the curve to the color- 
ist, who would know that the qualification curve is equal 30 
to one in the region between the transition zones. 

Color Correction Methodology 

[0096] FIG. 7 is a logic flow diagram illustrating a rou- 35 
tine 700 for operating the color correction equipment 80. 
Throughout the description of routine 700, the com- 
mands received by the color correction equipment 80 
are typically entered by a colorist using the various con- 
trols of the image processing system 10. In step 702, 40 
the color correction equipment 80 receives commands 
identifying the start and end frames of a scene of a video 
image to receive color correction. Also in step 702, the 
color correction equipment 80 receives a command se- 
lecting one of the frames of the scenes as a target frame 45 
to receive manual colorcorrections.Thecoior correction 
equipment 80 automatically applies the color correc- 
tions entered for the target frame to the other frames of 
the scene, as described below. 

[0097] Step 702 is followed by step 704, in which the 50 
color correction equipment 80 receives commands se- 
lecting a sample of the target frame. The sample is used 
as a starting point for defining a color correction region. 
Step 704 is followed by an optional step 706, in which 
the color correction equipment 80 may receive a com- 55 
mand defining a geometric constraint within the target 
frame. The geometric constraint is used to limit the ap- 
plication of color correction associated with the color 



correction region to a portion of the frames of the scene 
defined with respect to the geometric constraint. For ex- 
ample, color correction may only be applied on the in- 
side, or alternative on the outside, of the geometric con- 
straint within each frame of the scene. 
[0098] Step 706 is followed by another optional step 
708, in which the color correction equipment 80 receives 
a command defining an alpha filter. An alpha filter is 
qualification curve that modifies the total alpha curve to 
produce a final alpha qualification curve for the color cor- 
rection region. The total alpha curve is the product of 
the hue, saturation, and luminance qualification curve 
defined for the color correction region. The color correc- 
tion equipment 80 typically operates on a pixel-by-pixel 
basis by multiplying the alpha filter qualifier for a given 
pixel by the total alpha qualifier for the pixel to obtain 
the final alpha qualifier. The final alpha qualifier is then 
passed to an alpha mixer for use in blending the input 
color value for a the pixel with the corrected color value 
selected for the pixel by the colorist. Each of these qual- 
ification curves is typically trapezoidal in shape, al- 
though other filter shapes, such as square, bell-shaped, 
and so forth may be used. 

[0099] Step 708 is followed by step 71 0, in which the 
color correction equipment 80 receives commands de- 
fining the hue, saturation, and luminance qualification 
curves for the color correction region. The colorist may 
manipulate each of these qualification curves independ- 
ently. Step 71 0 is followed by step 71 2, in which the color 
correction equipment 80 receives commands defining 
the corrected color values forthe color correction region. 
Step 712 is followed by step 714, in which the color cor- 
rection equipment 80 computes the coefficients of a 
transformation matrix, known as the "T-matrix," which is 
used to produce an unqualified color corrected image. 
The alpha mixer mentioned above blends the color val- 
ues of an input image with the color values of an unqual- 
ified color corrected image in accordance with the final 
alpha qualification curve to product a qualified, color- 
corrected output image. The "T-matrix" and the alpha 
mixer are further described with reference to FIG. 12. 
The mathematics associated with the 'T-matrix" is de- 
scribed with reference to FIG. 13. 
[0100] Steps 704 through 712 provide the color cor- 
rection equipment 80 with information for applying color 
correction to a particular color correction region. These 
steps, of course, may be performed in a different order 
that is shown in routine 700, and a colorist my repeat 
any or all of the steps as desired. The color correction 
information created in steps 704 through 714 occupies 
one color correction channel of the color correction 
equipment 80. As described previously with reference 
to FIG. 6A, the color correction equipment 80 typically 
includes one or more color correction blocks that each 
provide up to sixteen non-overlapping color correction 
channels. Thus : the colorist may repeat step 704 
through 714 for additional color correction channels. 
[0101] Accordingly, in step 716, the color correction 
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equipment 80 receives commands indicating whether 
the colorist intends to enter color correction information 
for another color correction region, and thus occupy an- 
other color correction channel. If the colorist intends to 
enter color correction information for another color cor- 
rection region, the "YES" branch is followed from step 
716 to step 718, in which the color correction equipment 
80 determines whether another color correction channel 
is available. If another color correction channel is avail- 
able, the "YES" branch is followed from step 71 8 to step 
704, and the color correction equipment 80 receives 
color correction information for another color correction 
channel. 

[01 02] If the colorist does not intend to enter color cor- 
rection information for another color correction region, 
the "NO" branch is followed from step 716 to routine 720, 
in which the color correction equipment 80 applies the 
colorcorrections defined for the target frame to the other 
frames of the scene. Similarly, if there is not another 
color correction channel available in the color correction 
equipment 80, the "NO" branch is followed from step 
718 to routine 720, which is described below with refer- 
ence to FIG. 8. Routine 720 if followed by the "END" 
step, which preferably loops to step 702 so that routine 
700 may be repeated for another scene in the motion 
picture or other video program that the colorist is work- 
ing on. 

[0103] Routine 700 thus allows the color correction 
equipment 80 to receive color correction information for 
a number of color correction channel, each of which cor- 
responds to a non-overlapping color correction region 
defined in the hue domain. Each color correction region 
is ultimately defined by a final alpha qualification curve, 
which the color correction equipment 80 uses as a mix- 
ing function to blend the color values the input images 
with the color values of unqualified color corrected im- 
ages in the primary-color domain. Routine 700 may be 
repeated as needed to create a color corrected master 
film or video tape that reflects multiple color adjustments 
to multiple frames in multiple scenes. In the general 
case, multiple color adjustments may be applied to all 
of the frames of a motion picture or other video program. 
[0104] FIG. 8 is a logic flow diagram illustrating routine 
720 for automatically applying colorcorrections defined 
for a target frame of a scene to the other frames of the 
scene. Routine 720 follows the "NO" branches from 
steps 716 and 718 shown in FIG. 7. In step 802, the 
color correction equipment 80 loads look-up tables for 
the color correction regions defined by the colorist dur- 
ing routine 700. This is equivalentto saying that the color 
correction equipment 80 loads look-up tables for each 
color correction channel; and the color correction equip- 
ment 80 receives the data for the look-up tables through 
controls operated by the colorist, as described in routine 
700. 

[01 05] Step 802 is followed by step 804, in which the 
color correction equipment 80 configures the index gen- 
erator circuit. Step 804 is followed by step 806, in which 



the color correction equipment 80 gets the R,G,B color 
values for a frame of the scene to be color corrected. 
Step 806 is followed by step 808, in which the color cor- 
rection equipment 80 loads the color values for the 
5 frame into a frame buffer, from which the pixels will be 
passed, one by one, through color enhancement equip- 
ment, which is preferably constructed using program- 
mable logic units (PLUs), field-programmable gate ar- 
rays (FPGAs), or other types of reprogrammable paral- 

10 lel-processing hardware. Thus, the color enhancement 
equipment typically applies color correction on a pixel- 
by-pixel basis as the pixels pass through the circuitry of 
the color enhancement equipment. 
[0106] Step 808 is followed by step 810, in which the 

is color correction equipment 80 clocks a pixel from the 
frame buffer through the color enhancement equipment. 
Step 810 is followed by routine 812, in which the color 
correction equipment 80 computes a final alpha qualifier 
for the current pixel. Routine 812 is described with ref- 

20 erence to FIG. 9. Routine 812 is followed by step 814, 
in which the color correction equipment 80 blends the 
input color value for the current pixel with the corrected 
color value for the current pixel in accordance with the 
final alpha qualifiorforthe current pixel. That is, the color 

25 correction equipment 80 applies color correction to the 
primary-color domain value for the current pixel. This 
mixing operation, which is performed by the alpha mixer, 
is described with reference to FIG. 13A. 
[0107] Step 814 is followed by step 816, in which the 

30 color correction equipment 80 determines whetherthere 
is another pixel in the current frame to process. If there 
is another pixel in the current frame to process, the 
"YES" branch loops to step 810, in which the color cor- 
rection equipment 80 clocks the next pixel from the 

35 frame buffer through the color enhancement equipment. 
If there is not another pixel in the current frame to proc- 
ess, the "NO" branch is followed to step 818, in which 
the color correction equipment 80 determines whether 
there is another frame in the scene to process. If there 

40 js another frame in the scene to process, the "YES" 
branch loops to step 806, in which the color correction 
equipment 80 gets the color values for the next frame 
of the scene. If there is not another frame in the scene 
to process, the "NO" branch is followed to the "RE- 

^5 TURN" step 820, which returns to step 722 shown in 
FIG. 7. Routine 720 thus process each pixel of each 
frame of the scene. 

[0108] FIG. 9 is a logic flow diagram illustrating routine 
812 for computing a final alpha qualifier for the current 

50 pixel. Routine 812 follows step 810 shown in FIG. 8. As 
a prerequisite to routine 81 2, the color correction equip- 
ment 80 transforms the R,G,B primary-color-domain 
value for the pixel into an H,S,Y hue-domain color value. 
The color correction equipment 80 then performs the op- 

55 erations of routine 812 on the hue-domain color value. 
In other words, the color correction equipment 80 per- 
forms color correction region isolation, as defined by 
routine 812, in the hue domain. 
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determines the saturation qualifier (c^) for the pixel. And 
the luminance qualification curve for the pixel is passed 
from the luminance look-up table 1220 to the luminance 
ramp 1226, which determines the luminance qualifier 
(Oy) for the pixel. 

[0125] The hue qualifier (a h ), the saturation qualifier 
(ots), and the luminance qualifier (cty) are multiplied to 
produce a total alpha qualifier (o^), which is optionally 
passed to an alpha filter 1228. The alpha filter includes 
an additional qualification curve that may be defined 
with respect hue, saturation, or luminance. Accordingly, 
any or all of the hue-domain parameters (H,S,Y) for the 
given pixel may be passed to the alpha filter 1228. The 
alpha filter determines an alpha-filterqualifierforthegiv- 
en pixel that is multiplied by the total alpha qualifier (ofi 
to produce the final alpha qualifier (a). The final alpha 
qualifier (a) is then passed to an alpha mixer 1230. 
[0126] The user control interface 1214 also controls 
theconfiguration of acolor correction look-up table 1232 
that includes the coefficients of the T-matrix for the var- 
ious color correction channels. The index number is 
passed from the index generator 1212 to the color cor- 
rection look-up table 1232^ which passes the color cor- 
rection coefficients for the pixel's color correction chan- 
nel to a T-matrix multiplier array 1 234. The T-matrix mul- 
tiplier array 1234 also receives the input data signal 
1208, which includes the input color value (Rj n ,G jn ,B in ) 
for the given pixel. The T-matrix multiplier array 1234 
uses the input color value and the color correction co- 
efficients to compute the unqualified corrected color val- 
ue (R'.G'^.for the pixel. The T-matrix multiplier array 
1234 then passes the unqualified corrected color value 
to the alpha mixer 1 230. The mathematics and program- 
mable logic unit circuitry of the T-matrix multiplier array 
1234 is shown in FIG. 13B. 

[0127] The alpha mixer 1230 receives the input color 
value (R jnj Gj n ,B jn ), the unqualified corrected color value 
(R'^'.B*), and the final alpha qualifier (a) for the given 
pixel. The alpha mixer 1230 then computes the output 
color value (R out ,G out ,B out ) as a weighted sum of the in- 
put color value (R jn ,G jn ,B in ) and the unqualified correct- 
ed color value (R\G',B'), where final alpha qualifier (a) 
defines the weighting parameters, as shown in the fol- 
lowing alpha-mixing formula: 



above with two multipliers and one adder. In terms of 
programmable logic units, multipliers are more expen- 
sive than adders or subtracters. Therefore., it is advan- 
tageous to replace multipliers with adders or subtracters 

5 when possible. As shown in FIG. 1 3A, the alpha-mixing 
formula may be rearranged into a revised alpha-mixing 
formula that obtains the same result with one multipli- 
cation step and two subtraction steps. The alternative 
color mixer circuit 1 302 shows the circuit for implement- 

10 jng the revised alpha-mixing formula using programma- 
ble logic units. 

[0129] FIG. 13B is a schematic block diagram illus- 
trating the structure of a representative T-matrix multi- 
plication circuit 1306. The T-matrix multiplication circuit 
*5 1306 implements the T-matrix multiplication formula 
1304 shown in FIG. 13B using programmable logic 
units. 

[0130] FIG. 14 is a schematic block diagram illustrat- 
ing an index generator circuit 1212 implemented with 

20 programmable logic units. Each color correction chan- 
nel (i.e., hue sector) is defined by two comparison 
blocks. A first comparison block determines whether the 
hue parameter H for a given pixel is greater than the 
lower boundary of a corresponding color correction 

25 channel. And a second comparison block determines 
whether the hue parameter H for the pixel is less than 
or equal to the upper boundary of the color correction 
channel. If both conditions are satisfied, the pixel falls 
within the corresponding color correction channel, and 

30 an encoder returns the index number for the corre- 
sponding color correction channel. 
[0131] Thus, the present invention implements prima- 
ry color manipulation using secondary color correction 
region isolation. As such, the system performs color cor- 

35 rection in the primary color domain, which means that 
color data that is not manipulated passes through the 
system unaltered. In view of the foregoing description 
of the preferred embodiments of the invention, other em- 
bodiments will suggest themselves to those skilled in the 

to art. Therefore, the scope of the present invention is to 
be limited only by the claims set forth below. 



45 



R out = otR' + (1-a)Rin 



Claims 

1. An apparatus for selectively applying image 
processing to an image, comprising: 



G out = aG'+(1-a)G jn 



B 0Ut = aB' + (1-a)B jn 

[0128] FIG. 13A is a schematic block diagram illus- 
trating a color mixer circuit 1300 that may be implement- 
ed within the alpha mixer 1230. The color mixer circuit 
1300 implements the alpha mixing formula shown 



means for receiving an input signal correspond- 
ing to the image from an image source in a pri- 
mary color domain; 

means for receiving a command selecting a 
sample of the input signal; 
means for receiving a command identifying a 
color correction parameter in the primary color 
domain associated with the sample; 
means for determining a hue-domain parame- 
ter associated with the sample; 
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means defining a sector in the hue-domain 
about the hue-domain parameter associated 
with the sample; 

means defining a qualification curve corre- 
sponding to the sector; and 
a mixer operative for mixing the input signal and 
the color correction parameter within the sector 
in accordance with the qualification curve. 

2. The apparatus of Claim 1 , wherein: 

the hue-domain parameter is selected from the 
group including hue, saturation, and lumi- 
nance. 

3. The apparatus of Claim 1 , wherein the hue-domain 
parameter is a first hue-domain parameter and the 
qualification curve is a first qualification curve, and 
further comprising: 

means for determining a second hue-domain 
parameter associated with the sample; 
means defining a second qualification curve 
about the second hue-domain parameter and 
corresponding to the sector; 
means for combining the first and second qual- 
ification curves to obtain a combined qualifica- 
tion curve; and wherein 
the mixer is operative for mixing the input signal 
and the color correction parameter within the 
sector in accordance with the combined quali- 
fication curve. 

4. The apparatus of Claim 3, wherein: 

the first and second hue-domain parameters 
are selected from the group including hue, sat- 
uration, and luminance. 



5. The apparatus of Claim 3, further comprising: 

means for determining a third hue-domain pa- 
rameter associated with the sample; 
means defining a third qualification curve about 
the third hue-domain parameter and corre- 
sponding to the sector; 

means for combining the first, second, and third 
qualification curves to obtain a total qualifica- 
tion curve; and wherein 
the mixer is operative for mixing the input signal 
the color correction parameter within the sector 
in accordance with the total qualification curve. 

6. The apparatus of Claim 1 , wherein: 

the first, second, and third hue-domain param- 
eters include hue, saturation, and luminance. 
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7. The apparatus of Claim 5, further comprising: 

an alphas filter operative upon the total qualifi- 
cation curve for obtaining an alpha qualification 
curve; and wherein 

the mixer is operative for mixing the input signal 
and the color correction parameter within the 
sector in accordance with the alpha qualifica- 
tion curve. 

8. The apparatus of Claim 5 , further comprising: 

means defining a geometric constraint corre- 
sponding to the input signal; and 
means for limiting the mixing of the input signal 
and the color correction parameter in accord- 
ance with the geometric constraint. 



9. An apparatus for selectively applying image 
20 processing to images in a plurality of frames defin- 
ing a scene, comprising: 

means for receiving an input signal defining im- 
ages of a scene from an image source in a pri- 

25 mary color domain; 

means for receiving a command selecting a tar- 
get frame of the scene; 
means for receiving a command selecting a 
sample of the target frame; 

30 means for receiving a command identifying a 

color correction parameter in the primary color 
domain associated with the sample; 
means for determining a hue-domain parame- 
ter associated with the sample; 

35 means defining a sector in the hue-domain 

about the hue-domain parameter associated 
with the sample; 

means defining a qualification curve corre- 
sponding to the sector; and 
40 a mixer for mixing the input signal and the color 

correction parameter within the sector in ac- 
cordance with the qualification curve for each 
frame of the scene. 

45 10. The apparatus of Claim 9 , wherein: 

the hue-domain parameter is selected from the 
group including hue, saturation, and lumi- 
nance. 
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11. The apparatus of Claim 10, wherein the hue-do- 
main parameter is a first hue-domain parameter 
and the qualification curve is a first qualification 
curve, and further comprising: 

means for determining a second hue-domain 
parameter associated with the sample; 
means defining a second qualification curve 
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about the second hue-domain parameter and 
corresponding to the sector; 
means for combining the first and second qual- 
ification curves to obtain a combined qualifica- 
tion curve; and wherein 
the mixer is operative for mixing the input signal 
and the color correction parameter within the 
sector in accordance with the combined quali- 
fication curve. 

12. The apparatus of Claim 11 , wherein: 

the first and second hue-domain parameters 
are selected from the group including hue, sat- 
uration, and luminance. 

13. The apparatus of Claim 11, further comprising: 

means for determining a third hue-domain pa- 
rameter associated with the sample; 
means defining a third qualification curve about 
the third hue-domain parameter and corre- 
sponding to the sector; 

means for combining the first, second, and third 
qualification curves to obtain a total qualifica- 
tion curve; and wherein 
the mixer is operative for mixing the input signal 
and the color, correction parameter within the 
sector in accordance with the total qualification 
curve. 

14. The apparatus of Claim 13, wherein: 

the first, second, and third hue-domain param- 
eters include hue, saturation, and luminance. 

15. The apparatus of Claim 13, further comprising: 



a filter for filtering the total qualification curve to 

obtain an alpha qualification curve; and where- 
in 

the mixer is operative for mixing the input signal 
and the color correction parameter within the 
sector in accordance with the alpha qualifica- 
tion curve. 

16. The apparatus of Claim 13 further comprising: * 

means defining a geometric constraint corre- 
sponding to the input signal; and 
means for limiting the mixing of the input signal 
and the color correction parameter in accord- 
ance with the geometric constraint. 

1 7. A method for selectively applying image processing 
to an image, the method carried out in a system cou- 
pled to an image source, comprising the steps of: 
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receiving an input signal from the image 
source, the input signal corresponding to the 
image: 

receiving a color correction parameter for ap- 
plication to the image; 

determining a hue-domain parameter associat- 
ed with input signal; 

defining a sector in the hue-domain corre- 
sponding to the hue-domain parameter; 
defining a qualification curve corresponding to 
the sector; and 

mixing the input signal and the color correction 
parameter in accordance with the qualification 
curve. 

18. The method of Claim 1 7, wherein the input signal is 
in a primary color domain, and the mixing of the in- 
put signal and the color correction parameter is in 
conducted in the primary color domain. 

19. The method of Claim 17 : wherein: 

the hue-domain parameter is selected from the 
group including hue, saturation, and lumi- 
nance. 

20. The method of Claim 17 .further comprising the 
steps of: 

determining a second hue-domain parameter 
associated with the input signal; 
defining a second qualification curve about the 
second hue-domain parameter and corre- 
sponding to the sector; 

combining the first and second qualification 
curves to obtain a total qualification curve; and 
wherein 

the step of mixing the input signal comprises 
the step of mixing the input signal the color cor- ~ 
rection parameter within the sector in accord- 
ance with the total qualification curve. 

21. The method of Claim 20 : wherein: 

the first or second hue-domain parameters in- 
clude hue, saturation, and luminance. 

22. The method of Claim 20, further comprising the 
steps of: 

applying a filter to the total qualification curve 
to obtain an alpha qualification curve; and 
wherein 

the step of mixing the input signal comprises 
the step of mixing the input signal and the color 
correction parameter within the sector in ac- 
cordance with the alpha qualification curve. 
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23. The method of Claim 17, further comprising the 
steps of: 

defining a geometric constraint corresponding 
to the input signal; and 

mixing of the input signal and the color correc- 
tion parameter in accordance with the geomet- 
ric constraint. 

24. A method for selectively applying image processing 
to a plurality of images defining a scene, the proc- 
ess carried out in a system coupled to an image 
source, comprising the steps of: 

receiving an input signal corresponding to the 
images defining the scene from the image 
source; 

receiving a color correction parameter for ap- 
plication to the images; 

determining a hue-domain parameter associat- 
ed with the input signal; 
defining a sector in the hue-domain about the 
hue-domain parameter; 
defining a qualification curve corresponding to 
the sector; and 

for each image of the scene, mixing the input 
signal and the color correction parameter in ac- 
cordance with the qualification curve. 

25. The method of Claim 24, wherein: 

the hue-domain parameter is selected from the 
group including hue, saturation, and lumi- 
nance. 

26. The method of Claim 25, wherein the hue-domain 
parameter is a first hue-domain parameter and the 
qualification curve is a first qualification curve, fur- 

ther-comprising the steps of : 

determining a second hue-domain parameter 

associated with the sample; 

defining a second qualification curve about the 

second hue-domain parameter; 

combining the first and second qualification 

curves to obtain a total qualification curve; and 

wherein 

the step of mixing the input signal comprises 
the step of mixing the input signal and the color 
correction parameter in accordance with the 
combined total curve. 

27. The method of Claim 26, wherein: 

the first and second hue-domain parameters 
are selected from the group including hue, sat- 
uration, and luminance. 



28. The method of Claim 26, further comprising the 
steps of: 

applying a filter to the total qualification curve 
5 to obtain an alpha qualification curve; and 

wherein 

the step of mixing the input signal comprises 
the step of mixing the input signal and the color 
correction parameter in accordance with the al- 
10 pha qualification curve. 

29. The method of Claim 24, further comprising the 
steps of: 

15 defining a geometric constraint corresponding 

to the input signal; and 

mixing of the input signal and the color correc- 
tion parameter in accordance with the geomet- 
ric constraint. 

20 

30. A method for selectively applying image processing 
to an image, the method carried out i n a system cou- 
pled to an image source, comprising the steps of: 

25 receiving a primary domain input signal from 

the image source, the input signal correspond- 
ing to the image; 

receiving a color correction parameter for ap- 
plication to the image; 

30 defining a hue-domain qualifier for application 

of the color correction parameter; 
defining a geometric qualifier for application of 
the color correction parameter; and 
applying the color correction parameter to the 

35 input signal in accordance with the hue-domain 

qualifier and the geometric qualifier 

31 . The method of claim 30, wherein the input signal is 
in a primary color domain,and the application of the 

40 color correction parameter is effected in the primary 
color domain, without conversion to another color 
domain. 

32. The method of Claim 30, wherein the input signal is 
45 in a primary color domain, and the mixing of the in- 
put signal and the color correction parameter is in 
conducted in the primary color domain. 

33. The method of Claim 30 : wherein: 

50 

the hue-domain parameter is selected from the 
group including hue, saturation, and lumi- 
nance. 

55 34. The method of Claim 30, wherein the hue-domain 
parameter is a first hue-domain parameter, and fur- 
ther comprising the steps of: 
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determining a second hue-domain parameter 
associated with the input signal; 
defining a second qualification curve corre- 
sponding the second hue-domain parameter; 
combining the first and second qualification 
curves to obtain a total qualification curve; and 
wherein 

the step of applying the color correction param- 
eter to the input signal comprises applying the 
color correction parameter in accordance with 
the total qualification curve and the geometric 
qualifier. 

35. The method of Claim 30,wherein: 

the first or second hue-domain parameters in- 
clude hue, saturation, and luminance. 

36. The method of Claim 30, further comprising the 
steps of: 

applying a filter to the total qualification curve 
to obtain an alpha qualification curve; and 
wherein 

the step of applying the color correction param- 
eter to the input signal comprises applying the 
color correction parameter in accordance with 
the alpha qualification curve. 



the group including hue, saturation, and luminance. 

41. The system of Claim 37, wherein the hue-domain 
parameter is a first hue-domain parameter, and f ur- 

5 ther comprising; 

means for determining a second hue-domain 
parameter associated with the input signal; 
means defining a second qualification curve 
10 corresponding the second hue-domain param- 

eter; 

a mixer for combining the first and second qual- 
ification curves to obtain a total qualification 
curve; and 

15 wherein the component for applying the color 

correction parameter to the input signal is op- 
erative for applying the color correction param- 
eter to the input signal in accordance with the 
total qualification curve and the geometric qual- 

20 ifier. 

42. The system of Claim 37 wherein: 

the first or second hue-domain parameters in- 
25 elude hue, saturation, and luminance. 

43. The system of Claim 37, further comprising: 



37. A system for selectively applying image processing 
to an image, comprising: 

means for receiving a primary domain input sig- 
nal from an image source, the input signal cor- 
responding to the image; 
means for receiving a color correction parame- 
ter for application to the image; 
means defining a hue-domain qualifier for ap- 
plication of the color correction parameter; 
means defining a geometric qualifier for appli- 
cation of the color correction parameter; and 
a component for applying the color correction 
parameter to the input signal in accordance 
with the hue-domain qualifier and the geometric 
qualifier. 

38. The system of claim 37, wherein the input signal is 
in a primary color domain, and the application of the 
color correction parameter is effected in the primary 
color domain, without conversion to another color 
domain. 

39. The system of Claim 37, wherein the input signal is 
in a primary color domain, and the mixing of the in- 
put signal and the color correction parameter is in 
conducted in the primary color domain. 

40. The system of Claim 37, wherein: 

the hue-domain parameter is selected from 



a filter operative on the total qualification curve 
30 to obtain an alpha qualification curve; and 

wherein the component for applying the color 
correction parameter to the input signal is op- 
erative for applying the color correction param- 
eter in accordance with the alpha qualification 
35 curve. 
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